1 Goal

The goal here is to access datasets from NCEI. These datasets include daily, nonthly and hourly datasets.

This particular script uses the rNOAA dataset that accesses the servers the National Center for Environmental Information (NCEI)

A link to the way you do this with the web system is here. We’ll be using R to do this for us.

2 Libraries

The following libraries are needed for this exercise.

Tidyverse Packages

Other Packages


# Libraries

  # Tidyverse resources

  library(package = "tidyverse") # Multiple Tidyverse Resources
  library(package = "lubridate") # Date-Time Control

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date
  library(package = "ggridges")  # Ridgeline plots with ggplot2


  # NOAA Libraries

  library(package = "rnoaa") # NCEI  Data Retrieval Package
Registered S3 method overwritten by 'hoardr':
  method           from
  print.cache_info httr

3 Searching for Data

3.1. Available Datasets

The hardest part of this took is to look for the data. For our example there are 11 datasets to access using this tool. (rnoaa will let you get other data like storm and tornado data but we’re not doing that yet.) They are listed here using the ncdc_datasets() command.


# Use NCDC Datasets to gret available datasets

  ncdc_datasets()$data
NA
NA

To access these fields you can use the “id” value in the table above. So for daily summaries from the Global Historical Climate Data Network (GHCN) you would use “GHCND”.

3.2. Available Stations

Once you have dataset the fun begins with being able to find a given station or set of stations.

There are a couple ways to do this and they use ncdc_stations()

You can search by city, county, state, country, USGS watershed HUC number, zip code..

The different search methods are:


ncdc_locs_cats()$data
NA

The best way to search for a region is to use the web interface from NCEI/NCDC and get a “Location ID”

https://www.ncdc.noaa.gov/cdo-web/search

Use the “search for” and you should be able to pull from one of the above categories in the lower pull-down menu to get the code.

Set the limit to 1000 to make sure that you can get the total number of available of stations (so long as they are … othwewise you will get only 25 which is the default for the function)

For example:

3.2.1 Searching based on a city location

Stations within a somewhat arbitrary radius of the “CITY” of Sydney, Australia, can be found with “locationid” CITY:AS000010


# Station List for Sydney, Australia

ncdc_ids = ncdc_stations(locationid = 'CITY:AS000010', 
                         datasetid  = 'GHCND',
                         limit      = 1000)

ncdc_ids$data
NA

3.2.2 Searching based on a stations inside a given US County

Stations in Pennington “CNTY” can be found using “locatoinid” FIPS:46103 (sorry you can’t tunnel down to subdivisions in other countries like New South Wales, AU)


# Station List for Pennington County, SD

ncdc_ids = ncdc_stations(locationid = 'FIPS:46103', 
                         datasetid  = 'GHCND',
                         limit      = 1000)

ncdc_ids$data
NA

3.2.3 Searching based on a stations inside a given US State

Stations in the “ST” of South Dakota, FIPS:46


# Station List for South Dakota

ncdc_ids = ncdc_stations(locationid = 'FIPS:46', 
                         datasetid  = 'GHCND',
                         limit      = 1000)

ncdc_ids$data
NA

3.2.4 Searching based on a stations inside a given River Basin

Stations in the Rapid Creek Basin or other named USGS Stream Unit search for the “Hydrologic Accounting Unit” (HYD_ACC), HUC:10120110 (this only works for those basins in the US)’

(That station maked Lead 11S, or "11 km south of Lead is in the Rapid City Basin, not inside of Lead.)


# Station List for the Rapid Creek River Basin

ncdc_ids = ncdc_stations(locationid = 'HUC:10120110', 
                         datasetid  = 'GHCND',
                         limit      = 1000)

ncdc_ids$data
NA

3.2.5 Searching based on a stations inside a given country

Stations in North Korea that they allow us to see, FIPS:NK

(those two last stations that are in “South Korea” are radar stations in South Korea whose coverages cross the border into North Korea)


# Station List for the North Korea

ncdc_ids = ncdc_stations(locationid = 'FIPS:KN', 
                         limit      = 1000)

ncdc_ids$data
NA

4 Pulling In Data and Getting It Into Shape.

So let’s now try getting data from the Rapid City Airport.

Rapid City Airport (using the data from above for Pennington County, the GHCN station ID for the airport is… GHCND:USW00024090)

We can get the details for the station using the option “stationid = ‘GHCND:USW00024090’”


# Station Details for Rapid City Airport , SD

stationid_for_ncdcstations = 'GHCND:USW00024090'

stationid_for_ghcn_pull    = 'USW00024090'

# Station Details for Corvalis Or (OSU)


stationid_for_ncdcstations = 'GHCND:USC00351862'
stationid_for_ghcn_pull    = 'USC00351862'


# Station Details for Cheatah Reserve

stationid_for_ncdcstations = 'GHCND:WA009181280'
stationid_for_ghcn_pull    = 'WA009181280'




# Station Details for NRMCAS

stationid_for_ncdcstations = 'GHCND:USW00093727'
stationid_for_ghcn_pull    = 'USW00093727'

ncdc_ids = ncdc_stations(stationid = stationid_for_ncdcstations)

ncdc_ids = ncdc_ids$data

Also to get an inventory of the avaialble data for this station we can use the ncdc_datatypes() command.


# Get Available Parameters for a given station from a specific dataset

ncdc_datatypes(datasetid = 'GHCND',
               stationid = stationid_for_ncdcstations)$data
NA

4.1 Pulling the Raw Data from NCDC/NCEI

To pull the daily GHCN data for this station we use the meteo_tidy_ghcnd() function. We should be just able to ask, again, for the station ID for the airport.


# Pull the Raw Climate Data from a Single Station

ghcn_data = meteo_tidy_ghcnd(stationid  = stationid_for_ghcn_pull,
                             keep_flags = TRUE, 
                             var        = "all",
                             date_min   = NULL, 
                             date_max   = NULL)
file path:          /Users/wjc/Library/Caches/rnoaa/ghcnd/USW00093727.dly
file last updated:  2020-01-27 10:35:48
file min/max dates: 1955-01-01 / 2020-01-31
# drop empty variable columns (they should already have been dropped though)

ghcn_data = ghcn_data[,colSums(is.na(ghcn_data))<nrow(ghcn_data)]

ghcn_data
NA

The downside in the above output is that there are no units mentioned here. But they are available on the original data website here:

https://www1.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt

From the dataset the five basic parameters are

Code Parameter Name Units
PRCP Precipitation wrt 0000-0000 Local Time ⅒ of mm
SNOW Snowfall wrt 0000-0000 Local Time mm
SNWD Snow depth wrt 0000-0000 Local Time mm
TMAX Maximum temperature wrt 0000-0000 Local Time ⅒ of °C
TMIN Minimum temperature wrt 0000-0000 Local Time ⅒ of °C

And you may have

Code Parameter Name Units
TAVG Average temperature wrt 0000-0000 UTC ⅒ of °C

These really are the only parameters I normally pull.

I’m doing this by brute force and changing the temperature units to °F while I am at it…


# Changing Units




ghcn_data$tmax = ghcn_data$tmax/10 * 9. / 5. + 32
ghcn_data$tmin = ghcn_data$tmin/10 * 9. / 5. + 32

4.3. Patching any Missing Values

Here, we will do a check to ensure a continous dataset that places “NA” for missing records left_join() function


# create a date frame to set us up to accomodate missing data

daily_time_frame = tibble(date = seq.Date(from = min(ghcn_data$date),
                                          to   = max(ghcn_data$date), 
                                          by   = "1 day"))


ghcn_data = left_join(daily_time_frame,ghcn_data, by="date")

print(ghcn_data) 

remove(daily_time_frame)

And with that we can now [finally] play!

5 Make a Simple Plot

Let’s make a simple plot for Tmin


ggplot(data = ghcn_data) +   # use ghcn_data as the source of the data
  
  aes(x = date,              # select specific fields to plot
      y = tmin) +

  theme_bw() +              # use a very simple ploting theme
  
  ggtitle(label    = "Daily Global Historical Climate Data",
          subtitle = str_c(ncdc_ids$name,
                           " :: ",
                           stationid_for_ncdcstations)) + 
  
  xlab(label = "Date") +
  
  ylab(label = "Minimum Daily Temperature (°F)") +
  
  geom_line(color = "blue")  # make a simple line plot (and make it pink)

NA
NA

We can alsos take a close look at any flags for that parameter.


# get the tmin record in this case.

ghcn_tmin = ghcn_data %>% select(c("date",
                                   ends_with("tmin")))

ghcn_tmin %>% filter(qflag_tmin != " ")
NA

ggplot(data = ghcn_data) +   # use ghcn_data as the source of the data
  
  aes(x = date,              # select specific fields to plot
      y = tmax) +

  theme_bw() +              # use a very simple ploting theme
  
  ggtitle(label    = "Daily Global Historical Climate Data",
          subtitle = str_c(ncdc_ids$name,
                           " :: ",
                           stationid_for_ncdcstations)) + 
  
  xlab(label = "Date") +
  
  ylab(label = "Maximum Daily Temperature (°F)") +
  
  geom_line(color = "red")  # make a simple line plot (and make it pink)

NA
NA

We can alsos take a close look at any flags for that parameter.


# get the tmin record in this case.

ghcn_tmax = ghcn_data %>% select(c("date",
                                   ends_with("tmax")))

ghcn_tmax %>% filter(qflag_tmax != " ")
NA

6 Make a Fancy Plot to Show Changes in Climate

Now let’s fancy. Let’s see how max and min tempertures for each month changes over time.

Let’s chose some 30-y periods at the beginning and end of the record just as a demo. (You can add more periods if you want)

Let’s chose two parameters. Max and Minimum Temperatures

First, let’s change the data frame so that we have only min and max temperatures


# pull only the min and max temps

temp_data = ghcn_data %>% select(c(date, 
                                   tmax, 
                                   tmin))

We’ll start by creating an additional data column to represent the decade we’ll use the year() function to do this.


# Create a year parameter

temp_data$year = year(temp_data$date) 

# Turn it into a string that displays the start and end year of the decade

And add another variable for the calendar month


# add month

temp_data$month = month(x     = temp_data$date,
                        label = TRUE,
                        abbr  = TRUE)

And filter only the periods we want to use


# first full year (use the lowest date where the DOY via yday == 1)

period_1_year_start = year(min(temp_data$date[yday(temp_data$date)==1]))
period_1_year_end   = period_1_year_start + (30 - 1)

period_1_string     = str_c(period_1_year_start,
                            "-",
                            period_1_year_end,
                            sep = "")


# most recent year period

period_3_year_start = 2018 - (30 - 1)
period_3_year_end   = 2018
period_3_string     = str_c(period_3_year_start,
                            "-",
                            period_3_year_end,
                            sep = "")  

# a period in the middle

period_2_year_start = round((period_1_year_start + period_3_year_start) / 2 )
period_2_year_end   = round((period_1_year_end   + period_3_year_end) / 2 )
period_2_string     = str_c(period_2_year_start,
                            "-",
                            period_2_year_end,
                            sep = "")  
  

# extract these two periods

temp_data = temp_data %>% 
  filter(( (year(date) >= period_1_year_start)  & 
           (year(date) <= period_1_year_end   ) ) |
         ( (year(date) >= period_2_year_start)  & 
           (year(date) <= period_2_year_end   ) ) |
         ( (year(date) >= period_3_year_start)  & 
           (year(date) <= period_3_year_end   ) ) )

temp_data = temp_data %>% 
  mutate(Period = case_when((year(date) >= period_1_year_start)  & (year(date) <= period_1_year_end ) ~ period_1_string,
                            (year(date) >= period_2_year_start)  & (year(date) <= period_2_year_end ) ~ period_2_string,
                            (year(date) >= period_3_year_start)  & (year(date) <= period_3_year_end ) ~ period_3_string))

temp_data$Period = as.factor(temp_data$Period)

Make a ridge plot by month


# ridgeplot

ggplot(data = temp_data) +
  
    
  aes(x    = tmax,
      y    = fct_rev(month),
      fill = Period) +
  
  theme_bw() + 
  
    
  ggtitle(label    = "Daily Global Historical Climate Data",
          subtitle = str_c(ncdc_ids$name,
                           " :: ",
                           stationid_for_ncdcstations)) + 
  
    
  ylab(label = "Month") +
  
  xlab(label = "Daily Temperature (°F)") +
  
  

  scale_fill_manual(values=c("blue",
                             "yellow",
                             "red"))+

  
  geom_density_ridges2(alpha = 0.25,
                       color = NA) +

  geom_density_ridges2(mapping = aes(x    = tmin,
                                     y    = fct_rev(month),
                                     fill = Period),
                       alpha   = 0.25,
                       color   = NA) 

SCAVENGER HUNT

  1. Take the gchn data set
  2. Find Yearly Max-Max Temp, Min-Min Temp, Max-Daily Precip
  3. Try your method for getting max min returns

write.csv(x    =  ghcn_data, 
          file = str_c("./GHCN_",
                       unique(stationid_for_ghcn_pull),
                       ".csv"))
LS0tCnRpdGxlOiAiSW1wb3J0aW5nIERhaWx5IFN1bW1hcnkgRGF0YSBmcm9tIHRoZSBOYXRpb25hbCBDbGltYXRpYyBEYXRhIENlbnRlciIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKCiMgMSBHb2FsCgpUaGUgZ29hbCBoZXJlIGlzIHRvIGFjY2VzcyBkYXRhc2V0cyBmcm9tIE5DRUkuICBUaGVzZSBkYXRhc2V0cyBpbmNsdWRlIGRhaWx5LCBub250aGx5IGFuZCBob3VybHkgZGF0YXNldHMuICAKClRoaXMgcGFydGljdWxhciBzY3JpcHQgdXNlcyB0aGUgck5PQUEgZGF0YXNldCB0aGF0IGFjY2Vzc2VzIHRoZSBzZXJ2ZXJzIHRoZSBOYXRpb25hbCBDZW50ZXIgZm9yIEVudmlyb25tZW50YWwgSW5mb3JtYXRpb24gKE5DRUkpCgpBIGxpbmsgdG8gdGhlIHdheSB5b3UgZG8gdGhpcyB3aXRoIHRoZSB3ZWIgc3lzdGVtIGlzIFtoZXJlXShodHRwczovL3d3dy5uY2RjLm5vYWEuZ292L2Nkby13ZWIvd2Vic2VydmljZXMvdjIpLiAgV2UnbGwgYmUgdXNpbmcgUiB0byBkbyB0aGlzIGZvciB1cy4KCiMgMiAgTGlicmFyaWVzCgpUaGUgZm9sbG93aW5nIGxpYnJhcmllcyBhcmUgbmVlZGVkIGZvciB0aGlzIGV4ZXJjaXNlLgoKVGlkeXZlcnNlIFBhY2thZ2VzCgogICsgW3RpZHl2ZXJzZV0oaHR0cHM6Ly93d3cudGlkeXZlcnNlLm9yZykgOiBTZXQgb2YgY29tbW9ubHktdXNlZCBEYXRhIFNjaWVuY2UgcGFja2FnZXMgZm9yIFIgdGhhdCBpdCBjYW4gaW5zdGFsbCBhbmQgbG9hZCBhbGwgYXQgb25jZS4gSW4gdGhlIGxvbmctcnVuIHlvdSBwcm9iYWJseSBhbHNvIHdhbnQgdG8gaW5zdGFsbCB0aGUgdGlkeXZlcnNlIHBhY2thZ2Ugc3VpdGUgYW55d2F5LiBGb3IgdGhpcyBleGVyY2lzZSB0aGlzIHdpbGwgaW5jbHVkZS4uLiAgIAogICAgLSBbZ2dwbG90Ml0oaHR0cHM6Ly9nZ3Bsb3QyLnRpZHl2ZXJzZS5vcmcpIDogQ3JlYXRlIEVsZWdhbnQgRGF0YSBWaXN1YWxpemF0aW9ucyBVc2luZyB0aGUgR3JhbW1hciBvZiBHcmFwaGljcwogICAgLSBbdGliYmxlXShodHRwczovL3RpYmJsZS50aWR5dmVyc2Uub3JnKSA6IFNpbXBsZSBEYXRhIEZyYW1lcwogICAgLSBbdGlkeXJdKGh0dHBzOi8vdGlkeXIudGlkeXZlcnNlLm9yZykgOiBUb29scyBmb3Igc2hlcGhlcmRpbmcgZGF0YSBpbiBkYXRhIGZyYW1lcy4KICAgIC0gW3JlYWRyXShodHRwczovL3JlYWRyLnRpZHl2ZXJzZS5vcmcpIDogUmVhZCBSZWN0YW5ndWxhciBUZXh0IERhdGEKICAgIC0gW3B1cnJdKGh0dHBzOi8vcHVycnIudGlkeXZlcnNlLm9yZykgOiBGdW5jdGlvbmFsIFByb2dyYW1taW5nIFRvb2xzCiAgICAtIFtkcGx5cl0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnKSA6IEEgZ3JhbW1hciBvZiBkYXRhIG1hbmlwdWxhdGlvbgogICAgLSBbc3RyaW5ncl0oaHR0cHM6Ly9zdHJpbmdyLnRpZHl2ZXJzZS5vcmcpIDogU2ltcGxlLCBDb25zaXN0ZW50IFdyYXBwZXJzIGZvciBDb21tb24gU3RyaW5nIE9wZXJhdGlvbnMKICAgIC0gW2ZvcmNhdHNdKGh0dHBzOi8vZm9yY2F0cy50aWR5dmVyc2Uub3JnKSA6IFRvb2xzIGZvciBXb3JraW5nIHdpdGggQ2F0ZWdvcmljYWwgVmFyaWFibGVzIChGYWN0b3JzKQogICAgCiAgKyBbbHVicmlkYXRlXShodHRwczovL2x1YnJpZGF0ZS50aWR5dmVyc2Uub3JnKSA6IFRpbWUgYW5kIERhdGUgTWFuYWdlbWVudAogIAogICsgW2dncmlkZ2VzXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvZ2dyaWRnZXMvdmVyc2lvbnMvMC41LjEpIDogUmlkZ2VsaW5lIHBsb3RzIHdpdGggZ2dwbG90MgogIApPdGhlciBQYWNrYWdlcwoKICArIFtybm9hYV0oaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL3Jub2FhL3ZlcnNpb25zLzAuMi4wKSA6IENsaW1hdGUgRGF0YSBPbmxpbmUgU2VydmljZXMgZnJvbSBOQ0VJIAogICAgCmBgYHtyfQoKIyBMaWJyYXJpZXMKCiAgIyBUaWR5dmVyc2UgcmVzb3VyY2VzCgogIGxpYnJhcnkocGFja2FnZSA9ICJ0aWR5dmVyc2UiKSAjIE11bHRpcGxlIFRpZHl2ZXJzZSBSZXNvdXJjZXMKICBsaWJyYXJ5KHBhY2thZ2UgPSAibHVicmlkYXRlIikgIyBEYXRlLVRpbWUgQ29udHJvbAogIGxpYnJhcnkocGFja2FnZSA9ICJnZ3JpZGdlcyIpICAjIFJpZGdlbGluZSBwbG90cyB3aXRoIGdncGxvdDIKCgogICMgTk9BQSBMaWJyYXJpZXMKCiAgbGlicmFyeShwYWNrYWdlID0gInJub2FhIikgIyBOQ0VJICBEYXRhIFJldHJpZXZhbCBQYWNrYWdlCgpgYGAKCiMgMyBTZWFyY2hpbmcgZm9yIERhdGEKCiMjIDMuMS4gQXZhaWxhYmxlIERhdGFzZXRzCgpUaGUgaGFyZGVzdCBwYXJ0IG9mIHRoaXMgdG9vayBpcyB0byBsb29rIGZvciB0aGUgZGF0YS4gIEZvciBvdXIgZXhhbXBsZSB0aGVyZSBhcmUgMTEgZGF0YXNldHMgdG8gYWNjZXNzIHVzaW5nIHRoaXMgdG9vbC4gKHJub2FhIHdpbGwgbGV0IHlvdSBnZXQgb3RoZXIgZGF0YSBsaWtlIHN0b3JtIGFuZCB0b3JuYWRvIGRhdGEgYnV0IHdlJ3JlIG5vdCBkb2luZyB0aGF0IHlldC4pIFRoZXkgYXJlIGxpc3RlZCBoZXJlIHVzaW5nIHRoZSBbbmNkY19kYXRhc2V0cygpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL25jZGNfZGF0YXNldHMpIGNvbW1hbmQuCgpgYGB7cn0KCiMgVXNlIE5DREMgRGF0YXNldHMgdG8gZ3JldCBhdmFpbGFibGUgZGF0YXNldHMKCiAgbmNkY19kYXRhc2V0cygpJGRhdGEKICAKCmBgYAoKVG8gYWNjZXNzIHRoZXNlIGZpZWxkcyB5b3UgY2FuIHVzZSB0aGUgImlkIiB2YWx1ZSBpbiB0aGUgdGFibGUgYWJvdmUuICBTbyBmb3IgZGFpbHkgc3VtbWFyaWVzIGZyb20gdGhlIEdsb2JhbCBIaXN0b3JpY2FsIENsaW1hdGUgRGF0YSBOZXR3b3JrIChHSENOKSB5b3Ugd291bGQgdXNlICJHSENORCIuCgojIyAzLjIuICBBdmFpbGFibGUgU3RhdGlvbnMKCk9uY2UgeW91IGhhdmUgZGF0YXNldCB0aGUgZnVuIGJlZ2lucyB3aXRoIGJlaW5nIGFibGUgdG8gZmluZCBhIGdpdmVuIHN0YXRpb24gb3Igc2V0IG9mIHN0YXRpb25zLiAgCgpUaGVyZSBhcmUgYSBjb3VwbGUgd2F5cyB0byBkbyB0aGlzIGFuZCB0aGV5IHVzZSBbbmNkY19zdGF0aW9ucygpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL25jZGNfc3RhdGlvbnMpCgpZb3UgY2FuIHNlYXJjaCBieSBjaXR5LCBjb3VudHksIHN0YXRlLCBjb3VudHJ5LCBVU0dTIHdhdGVyc2hlZCBIVUMgbnVtYmVyLCB6aXAgY29kZS4uCgpUaGUgZGlmZmVyZW50IHNlYXJjaCBtZXRob2RzIGFyZToKCmBgYHtyfQoKbmNkY19sb2NzX2NhdHMoKSRkYXRhCgpgYGAKClRoZSBiZXN0IHdheSB0byBzZWFyY2ggZm9yIGEgcmVnaW9uIGlzIHRvIHVzZSB0aGUgd2ViIGludGVyZmFjZSBmcm9tIE5DRUkvTkNEQyBhbmQgZ2V0IGEgIkxvY2F0aW9uIElEIgoKW2h0dHBzOi8vd3d3Lm5jZGMubm9hYS5nb3YvY2RvLXdlYi9zZWFyY2hdKGh0dHBzOi8vd3d3Lm5jZGMubm9hYS5nb3YvY2RvLXdlYi9zZWFyY2gpCgpVc2UgdGhlICJzZWFyY2ggZm9yIiBhbmQgeW91IHNob3VsZCBiZSBhYmxlIHRvIHB1bGwgZnJvbSBvbmUgb2YgdGhlIGFib3ZlIGNhdGVnb3JpZXMgaW4gdGhlIGxvd2VyIHB1bGwtZG93biBtZW51IHRvIGdldCB0aGUgY29kZS4KCgoKU2V0IHRoZSBsaW1pdCB0byAxMDAwIHRvIG1ha2Ugc3VyZSB0aGF0IHlvdSBjYW4gZ2V0IHRoZSB0b3RhbCBudW1iZXIgb2YgYXZhaWxhYmxlIG9mIHN0YXRpb25zIChzbyBsb25nIGFzIHRoZXkgYXJlIC4uLiBvdGh3ZXdpc2UgeW91IHdpbGwgZ2V0IG9ubHkgMjUgd2hpY2ggaXMgdGhlIGRlZmF1bHQgZm9yIHRoZSBmdW5jdGlvbikKCkZvciBleGFtcGxlOgoKIyMjIDMuMi4xIFNlYXJjaGluZyBiYXNlZCBvbiBhIGNpdHkgbG9jYXRpb24KClN0YXRpb25zIHdpdGhpbiBhIHNvbWV3aGF0IGFyYml0cmFyeSByYWRpdXMgb2YgdGhlICJDSVRZIiBvZiBTeWRuZXksIEF1c3RyYWxpYSwgY2FuIGJlIGZvdW5kIHdpdGggImxvY2F0aW9uaWQiICoqQ0lUWTpBUzAwMDAxMCoqCgpgYGB7cn0KCiMgU3RhdGlvbiBMaXN0IGZvciBTeWRuZXksIEF1c3RyYWxpYQoKbmNkY19pZHMgPSBuY2RjX3N0YXRpb25zKGxvY2F0aW9uaWQgPSAnQ0lUWTpBUzAwMDAxMCcsIAogICAgICAgICAgICAgICAgICAgICAgICAgZGF0YXNldGlkICA9ICdHSENORCcsCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKIyMjIDMuMi4yIFNlYXJjaGluZyBiYXNlZCBvbiBhIHN0YXRpb25zIGluc2lkZSBhIGdpdmVuIFVTIENvdW50eQoKU3RhdGlvbnMgaW4gUGVubmluZ3RvbiAiQ05UWSIgY2FuIGJlIGZvdW5kIHVzaW5nICJsb2NhdG9pbmlkIiAqKkZJUFM6NDYxMDMqKiAoc29ycnkgeW91IGNhbid0IHR1bm5lbCBkb3duIHRvIHN1YmRpdmlzaW9ucyBpbiBvdGhlciBjb3VudHJpZXMgbGlrZSBOZXcgU291dGggV2FsZXMsIEFVKQoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgUGVubmluZ3RvbiBDb3VudHksIFNECgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOjQ2MTAzJywgCiAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhc2V0aWQgID0gJ0dIQ05EJywKICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWl0ICAgICAgPSAxMDAwKQoKbmNkY19pZHMkZGF0YQoKYGBgCgojIyMgMy4yLjMgU2VhcmNoaW5nIGJhc2VkIG9uIGEgc3RhdGlvbnMgaW5zaWRlIGEgZ2l2ZW4gVVMgU3RhdGUKClN0YXRpb25zIGluIHRoZSAiU1QiIG9mIFNvdXRoIERha290YSwgKipGSVBTOjQ2KiogIAoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgU291dGggRGFrb3RhCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOjQ2JywgCiAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhc2V0aWQgID0gJ0dIQ05EJywKICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWl0ICAgICAgPSAxMDAwKQoKbmNkY19pZHMkZGF0YQoKYGBgCgojIyMgMy4yLjQgU2VhcmNoaW5nIGJhc2VkIG9uIGEgc3RhdGlvbnMgaW5zaWRlIGEgZ2l2ZW4gUml2ZXIgQmFzaW4KClN0YXRpb25zIGluIHRoZSBSYXBpZCBDcmVlayBCYXNpbiBvciBvdGhlciBuYW1lZCBVU0dTIFN0cmVhbSBVbml0IHNlYXJjaCBmb3IgdGhlICJIeWRyb2xvZ2ljIEFjY291bnRpbmcgVW5pdCIgKEhZRF9BQ0MpLCAqKkhVQzoxMDEyMDExMCoqICh0aGlzIG9ubHkgd29ya3MgZm9yIHRob3NlIGJhc2lucyBpbiB0aGUgVVMpJwoKKFRoYXQgc3RhdGlvbiBtYWtlZCBMZWFkIDExUywgb3IgIjExIGttIHNvdXRoIG9mIExlYWQgaXMgaW4gdGhlIFJhcGlkIENpdHkgQmFzaW4sIG5vdCBpbnNpZGUgb2YgTGVhZC4pCgpgYGB7cn0KCiMgU3RhdGlvbiBMaXN0IGZvciB0aGUgUmFwaWQgQ3JlZWsgUml2ZXIgQmFzaW4KCm5jZGNfaWRzID0gbmNkY19zdGF0aW9ucyhsb2NhdGlvbmlkID0gJ0hVQzoxMDEyMDExMCcsIAogICAgICAgICAgICAgICAgICAgICAgICAgZGF0YXNldGlkICA9ICdHSENORCcsCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKIyMjIDMuMi41IFNlYXJjaGluZyBiYXNlZCBvbiBhIHN0YXRpb25zIGluc2lkZSBhIGdpdmVuIGNvdW50cnkKClN0YXRpb25zIGluIE5vcnRoIEtvcmVhIHRoYXQgdGhleSBhbGxvdyB1cyB0byBzZWUsICoqRklQUzpOSyoqICAKCih0aG9zZSB0d28gbGFzdCBzdGF0aW9ucyB0aGF0IGFyZSBpbiAiU291dGggS29yZWEiIGFyZSByYWRhciBzdGF0aW9ucyBpbiBTb3V0aCBLb3JlYSB3aG9zZSBjb3ZlcmFnZXMgY3Jvc3MgdGhlIGJvcmRlciBpbnRvIE5vcnRoIEtvcmVhKQoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgdGhlIE5vcnRoIEtvcmVhCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOktOJywgCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKCiMgNCBQdWxsaW5nIEluIERhdGEgYW5kIEdldHRpbmcgSXQgSW50byBTaGFwZS4KClNvIGxldCdzIG5vdyB0cnkgZ2V0dGluZyBkYXRhIGZyb20gdGhlIFJhcGlkIENpdHkgQWlycG9ydC4KClJhcGlkIENpdHkgQWlycG9ydCAodXNpbmcgdGhlIGRhdGEgZnJvbSBhYm92ZSBmb3IgUGVubmluZ3RvbiBDb3VudHksIHRoZSBHSENOIHN0YXRpb24gSUQgZm9yIHRoZSBhaXJwb3J0IGlzLi4uICoqR0hDTkQ6VVNXMDAwMjQwOTAqKikKCldlIGNhbiBnZXQgdGhlIGRldGFpbHMgZm9yIHRoZSBzdGF0aW9uIHVzaW5nIHRoZSBvcHRpb24gInN0YXRpb25pZCA9ICdHSENORDpVU1cwMDAyNDA5MCciCgpgYGB7cn0KCiMgU3RhdGlvbiBEZXRhaWxzIGZvciBSYXBpZCBDaXR5IEFpcnBvcnQgLCBTRAoKc3RhdGlvbmlkX2Zvcl9uY2Rjc3RhdGlvbnMgPSAnR0hDTkQ6VVNXMDAwMjQwOTAnCgpzdGF0aW9uaWRfZm9yX2doY25fcHVsbCAgICA9ICdVU1cwMDAyNDA5MCcKCiMgU3RhdGlvbiBEZXRhaWxzIGZvciBDb3J2YWxpcyBPciAoT1NVKQoKCnN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zID0gJ0dIQ05EOlVTQzAwMzUxODYyJwpzdGF0aW9uaWRfZm9yX2doY25fcHVsbCAgICA9ICdVU0MwMDM1MTg2MicKCgojIFN0YXRpb24gRGV0YWlscyBmb3IgQ2hlYXRhaCBSZXNlcnZlCgpzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucyA9ICdHSENORDpXQTAwOTE4MTI4MCcKc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwgICAgPSAnV0EwMDkxODEyODAnCgoKCgojIFN0YXRpb24gRGV0YWlscyBmb3IgTlJNQ0FTCgpzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucyA9ICdHSENORDpVU1cwMDA5MzcyNycKc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwgICAgPSAnVVNXMDAwOTM3MjcnCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMoc3RhdGlvbmlkID0gc3RhdGlvbmlkX2Zvcl9uY2Rjc3RhdGlvbnMpCgpuY2RjX2lkcyA9IG5jZGNfaWRzJGRhdGEKCmBgYAoKCgpBbHNvIHRvIGdldCBhbiBpbnZlbnRvcnkgb2YgdGhlIGF2YWlhbGJsZSBkYXRhIGZvciB0aGlzIHN0YXRpb24gd2UgY2FuIHVzZSB0aGUgW25jZGNfZGF0YXR5cGVzKCldKGh0dHBzOi8vd3d3LnJkb2N1bWVudGF0aW9uLm9yZy9wYWNrYWdlcy9ybm9hYS92ZXJzaW9ucy8wLjguNC90b3BpY3MvbmNkY19kYXRhdHlwZXMpIGNvbW1hbmQuCgpgYGB7cn0KCiMgR2V0IEF2YWlsYWJsZSBQYXJhbWV0ZXJzIGZvciBhIGdpdmVuIHN0YXRpb24gZnJvbSBhIHNwZWNpZmljIGRhdGFzZXQKCm5jZGNfZGF0YXR5cGVzKGRhdGFzZXRpZCA9ICdHSENORCcsCiAgICAgICAgICAgICAgIHN0YXRpb25pZCA9IHN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zKSRkYXRhCgpgYGAKCiMjIDQuMSBQdWxsaW5nIHRoZSBSYXcgRGF0YSBmcm9tIE5DREMvTkNFSQoKVG8gcHVsbCB0aGUgZGFpbHkgR0hDTiBkYXRhIGZvciB0aGlzIHN0YXRpb24gd2UgdXNlIHRoZSBbbWV0ZW9fdGlkeV9naGNuZCgpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL21ldGVvX3RpZHlfZ2hjbmQpIGZ1bmN0aW9uLiAgV2Ugc2hvdWxkIGJlIGp1c3QgYWJsZSB0byBhc2ssIGFnYWluLCBmb3IgdGhlIHN0YXRpb24gSUQgZm9yIHRoZSBhaXJwb3J0LgoKYGBge3J9CgojIFB1bGwgdGhlIFJhdyBDbGltYXRlIERhdGEgZnJvbSBhIFNpbmdsZSBTdGF0aW9uCgpnaGNuX2RhdGEgPSBtZXRlb190aWR5X2doY25kKHN0YXRpb25pZCAgPSBzdGF0aW9uaWRfZm9yX2doY25fcHVsbCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBrZWVwX2ZsYWdzID0gVFJVRSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmFyICAgICAgICA9ICJhbGwiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGVfbWluICAgPSBOVUxMLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkYXRlX21heCAgID0gTlVMTCkKCiMgZHJvcCBlbXB0eSB2YXJpYWJsZSBjb2x1bW5zICh0aGV5IHNob3VsZCBhbHJlYWR5IGhhdmUgYmVlbiBkcm9wcGVkIHRob3VnaCkKCmdoY25fZGF0YSA9IGdoY25fZGF0YVssY29sU3Vtcyhpcy5uYShnaGNuX2RhdGEpKTxucm93KGdoY25fZGF0YSldCgpnaGNuX2RhdGEKCmBgYAoKCgpUaGUgZG93bnNpZGUgaW4gdGhlIGFib3ZlIG91dHB1dCBpcyB0aGF0IHRoZXJlIGFyZSBubyB1bml0cyBtZW50aW9uZWQgaGVyZS4gIEJ1dCB0aGV5ICphcmUqIGF2YWlsYWJsZSBvbiB0aGUgb3JpZ2luYWwgZGF0YSB3ZWJzaXRlIGhlcmU6CgpbaHR0cHM6Ly93d3cxLm5jZGMubm9hYS5nb3YvcHViL2RhdGEvZ2hjbi9kYWlseS9yZWFkbWUudHh0XShodHRwczovL3d3dzEubmNkYy5ub2FhLmdvdi9wdWIvZGF0YS9naGNuL2RhaWx5L3JlYWRtZS50eHQpCgpGcm9tIHRoZSBkYXRhc2V0IHRoZSBmaXZlIGJhc2ljIHBhcmFtZXRlcnMgYXJlIAoKfCBDb2RlIHwgUGFyYW1ldGVyIE5hbWUgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCBVbml0cyAgICB8CnwtLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tfAp8IFBSQ1AgfCBQcmVjaXBpdGF0aW9uIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICB8IOKFkiBvZiBtbSB8CnwgU05PVyB8IFNub3dmYWxsIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICAgICAgIHwgbW0gICAgICAgfAp8IFNOV0QgfCBTbm93IGRlcHRoIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICAgICB8IG1tICAgICAgIHwKfCBUTUFYIHwgTWF4aW11bSB0ZW1wZXJhdHVyZSB3cnQgMDAwMC0wMDAwIExvY2FsIFRpbWUgfCDihZIgb2YgwrBDIHwKfCBUTUlOIHwgTWluaW11bSB0ZW1wZXJhdHVyZSB3cnQgMDAwMC0wMDAwIExvY2FsIFRpbWUgfCDihZIgb2YgwrBDIHwKCkFuZCB5b3UgbWF5IGhhdmUgCgp8IENvZGUgfCBQYXJhbWV0ZXIgTmFtZSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8IFVuaXRzICAgIHwKfC0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS18CnwgVEFWRyB8IEF2ZXJhZ2UgdGVtcGVyYXR1cmUgd3J0IDAwMDAtMDAwMCBVVEMgICAgICAgIHwg4oWSIG9mIMKwQyB8CgpUaGVzZSByZWFsbHkgYXJlIHRoZSBvbmx5IHBhcmFtZXRlcnMgSSBub3JtYWxseSBwdWxsLgoKCkknbSBkb2luZyB0aGlzIGJ5IGJydXRlIGZvcmNlIGFuZCBjaGFuZ2luZyB0aGUgdGVtcGVyYXR1cmUgdW5pdHMgdG8gwrBGIHdoaWxlIEkgYW0gYXQgaXQuLi4gIAoKYGBge3J9CgojIENoYW5naW5nIFVuaXRzCgoKCgpnaGNuX2RhdGEkdG1heCA9IGdoY25fZGF0YSR0bWF4LzEwICogOS4gLyA1LiArIDMyCmdoY25fZGF0YSR0bWluID0gZ2hjbl9kYXRhJHRtaW4vMTAgKiA5LiAvIDUuICsgMzIKCgpgYGAKCiMjIDQuMy4gUGF0Y2hpbmcgYW55IE1pc3NpbmcgVmFsdWVzCgpIZXJlLCB3ZSB3aWxsIGRvIGEgY2hlY2sgdG8gZW5zdXJlIGEgY29udGlub3VzIGRhdGFzZXQgdGhhdCBwbGFjZXMgIk5BIiBmb3IgbWlzc2luZyByZWNvcmRzIFtsZWZ0X2pvaW4oKV0oaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL2Jhc2UvdmVyc2lvbnMvMy4wLjMvdG9waWNzL2NiaW5kKSBmdW5jdGlvbgoKYGBge3J9CgojIGNyZWF0ZSBhIGRhdGUgZnJhbWUgdG8gc2V0IHVzIHVwIHRvIGFjY29tb2RhdGUgbWlzc2luZyBkYXRhCgpkYWlseV90aW1lX2ZyYW1lID0gdGliYmxlKGRhdGUgPSBzZXEuRGF0ZShmcm9tID0gbWluKGdoY25fZGF0YSRkYXRlKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdG8gICA9IG1heChnaGNuX2RhdGEkZGF0ZSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBieSAgID0gIjEgZGF5IikpCgoKZ2hjbl9kYXRhID0gbGVmdF9qb2luKGRhaWx5X3RpbWVfZnJhbWUsZ2hjbl9kYXRhLCBieT0iZGF0ZSIpCgpwcmludChnaGNuX2RhdGEpIAoKcmVtb3ZlKGRhaWx5X3RpbWVfZnJhbWUpCgpgYGAKCgpBbmQgd2l0aCB0aGF0IHdlIGNhbiBub3cgW2ZpbmFsbHldIHBsYXkhCgojIDUgTWFrZSBhIFNpbXBsZSBQbG90CgpMZXQncyBtYWtlIGEgc2ltcGxlIHBsb3QgZm9yIFRtaW4KCmBgYHtyfQoKZ2dwbG90KGRhdGEgPSBnaGNuX2RhdGEpICsgICAjIHVzZSBnaGNuX2RhdGEgYXMgdGhlIHNvdXJjZSBvZiB0aGUgZGF0YQogIAogIGFlcyh4ID0gZGF0ZSwgICAgICAgICAgICAgICMgc2VsZWN0IHNwZWNpZmljIGZpZWxkcyB0byBwbG90CiAgICAgIHkgPSB0bWluKSArCgogIHRoZW1lX2J3KCkgKyAgICAgICAgICAgICAgIyB1c2UgYSB2ZXJ5IHNpbXBsZSBwbG90aW5nIHRoZW1lCiAgCiAgZ2d0aXRsZShsYWJlbCAgICA9ICJEYWlseSBHbG9iYWwgSGlzdG9yaWNhbCBDbGltYXRlIERhdGEiLAogICAgICAgICAgc3VidGl0bGUgPSBzdHJfYyhuY2RjX2lkcyRuYW1lLAogICAgICAgICAgICAgICAgICAgICAgICAgICAiIDo6ICIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zKSkgKyAKICAKICB4bGFiKGxhYmVsID0gIkRhdGUiKSArCiAgCiAgeWxhYihsYWJlbCA9ICJNaW5pbXVtIERhaWx5IFRlbXBlcmF0dXJlICjCsEYpIikgKwogIAogIGdlb21fbGluZShjb2xvciA9ICJibHVlIikgICMgbWFrZSBhIHNpbXBsZSBsaW5lIHBsb3QgKGFuZCBtYWtlIGl0IHBpbmspCgoKYGBgCgpXZSBjYW4gYWxzb3MgdGFrZSBhIGNsb3NlIGxvb2sgYXQgYW55IGZsYWdzIGZvciB0aGF0IHBhcmFtZXRlci4KCgpgYGB7cn0KCiMgZ2V0IHRoZSB0bWluIHJlY29yZCBpbiB0aGlzIGNhc2UuCgpnaGNuX3RtaW4gPSBnaGNuX2RhdGEgJT4lIHNlbGVjdChjKCJkYXRlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbmRzX3dpdGgoInRtaW4iKSkpCgpnaGNuX3RtaW4gJT4lIGZpbHRlcihxZmxhZ190bWluICE9ICIgIikKCmBgYApgYGB7cn0KCmdncGxvdChkYXRhID0gZ2hjbl9kYXRhKSArICAgIyB1c2UgZ2hjbl9kYXRhIGFzIHRoZSBzb3VyY2Ugb2YgdGhlIGRhdGEKICAKICBhZXMoeCA9IGRhdGUsICAgICAgICAgICAgICAjIHNlbGVjdCBzcGVjaWZpYyBmaWVsZHMgdG8gcGxvdAogICAgICB5ID0gdG1heCkgKwoKICB0aGVtZV9idygpICsgICAgICAgICAgICAgICMgdXNlIGEgdmVyeSBzaW1wbGUgcGxvdGluZyB0aGVtZQogIAogIGdndGl0bGUobGFiZWwgICAgPSAiRGFpbHkgR2xvYmFsIEhpc3RvcmljYWwgQ2xpbWF0ZSBEYXRhIiwKICAgICAgICAgIHN1YnRpdGxlID0gc3RyX2MobmNkY19pZHMkbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIiA6OiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICBzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucykpICsgCiAgCiAgeGxhYihsYWJlbCA9ICJEYXRlIikgKwogIAogIHlsYWIobGFiZWwgPSAiTWF4aW11bSBEYWlseSBUZW1wZXJhdHVyZSAowrBGKSIpICsKICAKICBnZW9tX2xpbmUoY29sb3IgPSAicmVkIikgICMgbWFrZSBhIHNpbXBsZSBsaW5lIHBsb3QgKGFuZCBtYWtlIGl0IHBpbmspCgoKYGBgCgpXZSBjYW4gYWxzb3MgdGFrZSBhIGNsb3NlIGxvb2sgYXQgYW55IGZsYWdzIGZvciB0aGF0IHBhcmFtZXRlci4KCgpgYGB7cn0KCiMgZ2V0IHRoZSB0bWluIHJlY29yZCBpbiB0aGlzIGNhc2UuCgpnaGNuX3RtYXggPSBnaGNuX2RhdGEgJT4lIHNlbGVjdChjKCJkYXRlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbmRzX3dpdGgoInRtYXgiKSkpCgpnaGNuX3RtYXggJT4lIGZpbHRlcihxZmxhZ190bWF4ICE9ICIgIikKCmBgYAoKIyA2IE1ha2UgYSBGYW5jeSBQbG90IHRvIFNob3cgQ2hhbmdlcyBpbiBDbGltYXRlCgpOb3cgbGV0J3MgZmFuY3kuICBMZXQncyBzZWUgaG93IG1heCBhbmQgbWluIHRlbXBlcnR1cmVzIGZvciBlYWNoIG1vbnRoIGNoYW5nZXMgb3ZlciB0aW1lLgoKTGV0J3MgY2hvc2Ugc29tZSAzMC15IHBlcmlvZHMgYXQgdGhlIGJlZ2lubmluZyBhbmQgZW5kIG9mIHRoZSByZWNvcmQganVzdCBhcyBhIGRlbW8uICAoWW91IGNhbiBhZGQgbW9yZSBwZXJpb2RzIGlmIHlvdSB3YW50KQoKTGV0J3MgY2hvc2UgdHdvIHBhcmFtZXRlcnMuIE1heCBhbmQgTWluaW11bSBUZW1wZXJhdHVyZXMKCkZpcnN0LCBsZXQncyBjaGFuZ2UgdGhlIGRhdGEgZnJhbWUgc28gdGhhdCB3ZSBoYXZlIG9ubHkgbWluIGFuZCBtYXggdGVtcGVyYXR1cmVzCgpgYGB7cn0KCiMgcHVsbCBvbmx5IHRoZSBtaW4gYW5kIG1heCB0ZW1wcwoKdGVtcF9kYXRhID0gZ2hjbl9kYXRhICU+JSBzZWxlY3QoYyhkYXRlLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0bWF4LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0bWluKSkKCmBgYAoKCldlJ2xsIHN0YXJ0IGJ5IGNyZWF0aW5nIGFuIGFkZGl0aW9uYWwgZGF0YSBjb2x1bW4gdG8gcmVwcmVzZW50IHRoZSBkZWNhZGUgd2UnbGwgdXNlIHRoZSBbeWVhcigpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvbHVicmlkYXRlL3ZlcnNpb25zLzEuNy40L3RvcGljcy95ZWFyKSBmdW5jdGlvbiB0byBkbyB0aGlzLgoKYGBge3J9CgojIENyZWF0ZSBhIHllYXIgcGFyYW1ldGVyCgp0ZW1wX2RhdGEkeWVhciA9IHllYXIodGVtcF9kYXRhJGRhdGUpIAoKIyBUdXJuIGl0IGludG8gYSBzdHJpbmcgdGhhdCBkaXNwbGF5cyB0aGUgc3RhcnQgYW5kIGVuZCB5ZWFyIG9mIHRoZSBkZWNhZGUKCmBgYAoKQW5kIGFkZCBhbm90aGVyIHZhcmlhYmxlIGZvciB0aGUgY2FsZW5kYXIgbW9udGgKCmBgYHtyfQoKIyBhZGQgbW9udGgKCnRlbXBfZGF0YSRtb250aCA9IG1vbnRoKHggICAgID0gdGVtcF9kYXRhJGRhdGUsCiAgICAgICAgICAgICAgICAgICAgICAgIGxhYmVsID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgICAgYWJiciAgPSBUUlVFKQoKYGBgCgoKQW5kIGZpbHRlciBvbmx5IHRoZSBwZXJpb2RzIHdlIHdhbnQgdG8gdXNlCgpgYGB7cn0KCiMgZmlyc3QgZnVsbCB5ZWFyICh1c2UgdGhlIGxvd2VzdCBkYXRlIHdoZXJlIHRoZSBET1kgdmlhIHlkYXkgPT0gMSkKCnBlcmlvZF8xX3llYXJfc3RhcnQgPSB5ZWFyKG1pbih0ZW1wX2RhdGEkZGF0ZVt5ZGF5KHRlbXBfZGF0YSRkYXRlKT09MV0pKQpwZXJpb2RfMV95ZWFyX2VuZCAgID0gcGVyaW9kXzFfeWVhcl9zdGFydCArICgzMCAtIDEpCgpwZXJpb2RfMV9zdHJpbmcgICAgID0gc3RyX2MocGVyaW9kXzFfeWVhcl9zdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICItIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlcmlvZF8xX3llYXJfZW5kLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc2VwID0gIiIpCgoKIyBtb3N0IHJlY2VudCB5ZWFyIHBlcmlvZAoKcGVyaW9kXzNfeWVhcl9zdGFydCA9IDIwMTggLSAoMzAgLSAxKQpwZXJpb2RfM195ZWFyX2VuZCAgID0gMjAxOApwZXJpb2RfM19zdHJpbmcgICAgID0gc3RyX2MocGVyaW9kXzNfeWVhcl9zdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICItIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlcmlvZF8zX3llYXJfZW5kLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc2VwID0gIiIpICAKCiMgYSBwZXJpb2QgaW4gdGhlIG1pZGRsZQoKcGVyaW9kXzJfeWVhcl9zdGFydCA9IHJvdW5kKChwZXJpb2RfMV95ZWFyX3N0YXJ0ICsgcGVyaW9kXzNfeWVhcl9zdGFydCkgLyAyICkKcGVyaW9kXzJfeWVhcl9lbmQgICA9IHJvdW5kKChwZXJpb2RfMV95ZWFyX2VuZCAgICsgcGVyaW9kXzNfeWVhcl9lbmQpIC8gMiApCnBlcmlvZF8yX3N0cmluZyAgICAgPSBzdHJfYyhwZXJpb2RfMl95ZWFyX3N0YXJ0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIi0iLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgcGVyaW9kXzJfeWVhcl9lbmQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBzZXAgPSAiIikgIAogIAoKIyBleHRyYWN0IHRoZXNlIHR3byBwZXJpb2RzCgp0ZW1wX2RhdGEgPSB0ZW1wX2RhdGEgJT4lIAogIGZpbHRlcigoICh5ZWFyKGRhdGUpID49IHBlcmlvZF8xX3llYXJfc3RhcnQpICAmIAogICAgICAgICAgICh5ZWFyKGRhdGUpIDw9IHBlcmlvZF8xX3llYXJfZW5kICAgKSApIHwKICAgICAgICAgKCAoeWVhcihkYXRlKSA+PSBwZXJpb2RfMl95ZWFyX3N0YXJ0KSAgJiAKICAgICAgICAgICAoeWVhcihkYXRlKSA8PSBwZXJpb2RfMl95ZWFyX2VuZCAgICkgKSB8CiAgICAgICAgICggKHllYXIoZGF0ZSkgPj0gcGVyaW9kXzNfeWVhcl9zdGFydCkgICYgCiAgICAgICAgICAgKHllYXIoZGF0ZSkgPD0gcGVyaW9kXzNfeWVhcl9lbmQgICApICkgKQoKdGVtcF9kYXRhID0gdGVtcF9kYXRhICU+JSAKICBtdXRhdGUoUGVyaW9kID0gY2FzZV93aGVuKCh5ZWFyKGRhdGUpID49IHBlcmlvZF8xX3llYXJfc3RhcnQpICAmICh5ZWFyKGRhdGUpIDw9IHBlcmlvZF8xX3llYXJfZW5kICkgfiBwZXJpb2RfMV9zdHJpbmcsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAoeWVhcihkYXRlKSA+PSBwZXJpb2RfMl95ZWFyX3N0YXJ0KSAgJiAoeWVhcihkYXRlKSA8PSBwZXJpb2RfMl95ZWFyX2VuZCApIH4gcGVyaW9kXzJfc3RyaW5nLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgKHllYXIoZGF0ZSkgPj0gcGVyaW9kXzNfeWVhcl9zdGFydCkgICYgKHllYXIoZGF0ZSkgPD0gcGVyaW9kXzNfeWVhcl9lbmQgKSB+IHBlcmlvZF8zX3N0cmluZykpCgp0ZW1wX2RhdGEkUGVyaW9kID0gYXMuZmFjdG9yKHRlbXBfZGF0YSRQZXJpb2QpCgpgYGAKCgpNYWtlIGEgcmlkZ2UgcGxvdCBieSBtb250aAoKCmBgYHtyfQoKIyByaWRnZXBsb3QKCmdncGxvdChkYXRhID0gdGVtcF9kYXRhKSArCiAgCiAgICAKICBhZXMoeCAgICA9IHRtYXgsCiAgICAgIHkgICAgPSBmY3RfcmV2KG1vbnRoKSwKICAgICAgZmlsbCA9IFBlcmlvZCkgKwogIAogIHRoZW1lX2J3KCkgKyAKICAKICAgIAogIGdndGl0bGUobGFiZWwgICAgPSAiRGFpbHkgR2xvYmFsIEhpc3RvcmljYWwgQ2xpbWF0ZSBEYXRhIiwKICAgICAgICAgIHN1YnRpdGxlID0gc3RyX2MobmNkY19pZHMkbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIiA6OiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICBzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucykpICsgCiAgCiAgICAKICB5bGFiKGxhYmVsID0gIk1vbnRoIikgKwogIAogIHhsYWIobGFiZWwgPSAiRGFpbHkgVGVtcGVyYXR1cmUgKMKwRikiKSArCiAgCiAgCgogIHNjYWxlX2ZpbGxfbWFudWFsKHZhbHVlcz1jKCJibHVlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAieWVsbG93IiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAicmVkIikpKwoKICAKICBnZW9tX2RlbnNpdHlfcmlkZ2VzMihhbHBoYSA9IDAuMjUsCiAgICAgICAgICAgICAgICAgICAgICAgY29sb3IgPSBOQSkgKwoKICBnZW9tX2RlbnNpdHlfcmlkZ2VzMihtYXBwaW5nID0gYWVzKHggICAgPSB0bWluLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgeSAgICA9IGZjdF9yZXYobW9udGgpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmlsbCA9IFBlcmlvZCksCiAgICAgICAgICAgICAgICAgICAgICAgYWxwaGEgICA9IDAuMjUsCiAgICAgICAgICAgICAgICAgICAgICAgY29sb3IgICA9IE5BKSAKCmBgYAoKIyBTQ0FWRU5HRVIgSFVOVAoKMSkgVGFrZSB0aGUgZ2NobiBkYXRhIHNldAoyKSBGaW5kIFllYXJseSBNYXgtTWF4IFRlbXAsIE1pbi1NaW4gVGVtcCwgTWF4LURhaWx5IFByZWNpcAozKSBUcnkgeW91ciBtZXRob2QgZm9yIGdldHRpbmcgbWF4IG1pbiByZXR1cm5zIAoKYGBge3J9Cgp3cml0ZS5jc3YoeCAgICA9ICBnaGNuX2RhdGEsIAogICAgICAgICAgZmlsZSA9IHN0cl9jKCIuL0dIQ05fIiwKICAgICAgICAgICAgICAgICAgICAgICB1bmlxdWUoc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwpLAogICAgICAgICAgICAgICAgICAgICAgICIuY3N2IikpCmBgYAoK